Why I'm doing:

Primary key table recover is used for the case that PK table meet inconsistent issue, error message like:

"insert found duplicate key"
"unexpected segment id"
"delvec inconsistent"

and we need to auto recover from this issue by rebuild PK index and delete vector.
But in current implementation, it doesn't support:

1. recognize deletion files
2. cloud native index rebuild

So in this PR we want to fix these.

What I'm doing:

This pull request includes several enhancements and bug fixes related to the primary key recovery mechanism in the shared-data mode. The changes include improvements to PK table recover capability, the addition of test hooks for recovery scenarios, and updates to unit tests to cover new recovery cases.

Enhancements to metadata handling:

• be/src/storage/lake/lake_primary_key_recover.cpp: Added handling for SSTable metadata in the pre_cleanup method and improved iterator handling for del files during rowset iteration. [1] [2]

Addition of test hooks:

• be/src/storage/lake/meta_file.cpp: Introduced a test hook in MetaFileBuilder::update_num_del_stat to set a recovery flag for testing recovery mode.
• be/src/storage/lake/update_manager.cpp: Added a test hook in UpdateManager::publish_primary_key_tablet to set a recovery flag for testing recovery mode.

Updates to unit tests:

• be/test/storage/lake/primary_key_publish_test.cpp: Expanded the LakePrimaryKeyPublishTest with new test cases to cover various recovery scenarios, including recovery with multiple reasons and handling of del files. [1] [2]

What type of PR is this:

• BugFix
• Feature
• Enhancement
• Refactor
• UT
• Doc
• Tool

Does this PR entail a change in behavior?

• Yes, this PR will result in a change in behavior.
• No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

• Interface/UI changes: syntax, type conversion, expression evaluation, display information
• Parameter changes: default values, similar parameters but with different default values
• Policy changes: use new policy to replace old one, functionality automatically enabled
• Feature removed
• Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

• I have added test cases for my bug fix or my new feature
• This pr needs user documentation (for new or modified features or behaviors)
  • I have added documentation for my new feature or new function
• This is a backport pr

Bugfix cherry-pick branch check:

• I have checked the version labels which the pr will be auto-backported to the target branch
  • 3.3
  • 3.2
  • 3.1
  • 3.0
  • 2.5